Systems and methods for providing vibration feedback in robotic systems

ABSTRACT

Systems and methods for training an operator of a robotic surgery system are disclosed. One such method includes enabling the operator to perform a test procedure, recording vibrations of a surgical tool of the robotic surgery system during the test procedure, and generating a score for the operator based at least in part on the recorded vibrations of the surgical tool.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 13/577,581, filed Aug. 7, 2012, which is the U.S. NationalPhase application of PCT International Application No.PCT/US2011/023995, filed Feb. 8, 2011, which claims priority to U.S.Patent Application No. 61/302,681, filed Feb. 9, 2010, each of whichapplications are incorporated herein by reference in their entiretiesand for all purposes.

STATEMENT OF FEDERALLY SPONSORED RESEARCH

This invention was made with government support under Grant No.IIS-0845670 awarded by the National Science Foundation, and under GrantNo. UL1TR000003 awarded by the National Institutes of Health. Thegovernment may have certain rights in the invention.

FIELD OF THE INVENTION

The present invention relates generally to robotic systems, and moreparticularly to providing vibration feedback for users of roboticsystems.

BACKGROUND OF THE INVENTION

Generally, robotic systems are useful for performing tasks that humansare otherwise unable or unwilling to perform. Robotic systems may beparticularly useful for performing tasks that require a strength,dexterity, size, or visualization that humans cannot easily replicate.One example of such a robotic system is a robotic surgery system, i.e.,a robotic system that assists surgeons in performing surgicalprocedures.

For certain tasks, it may be desirable that the robotic system include arobotic component that is remotely controlled by a human operator. Ahuman operator may control the robotic component in its performance ofthe desired task. However, certain tasks, e.g. surgeries, may requirevery precise movements by the robotic component, and therefore, veryprecise control by the operator of the robotic system. To achieve therequired control, the human operator may rely on visual feedback, i.e.,watching the robotic component, to perform the task. Nonetheless,improved robotic systems are desired for performing all types of tasks.

SUMMARY OF THE INVENTION

Aspects of the present invention are related to systems and methods forproviding vibration feedback in robotic systems.

In accordance with one aspect of the present invention, a roboticsurgery system includes an armature, a control station, a sensor, anactuator, and a controller. The armature is configured to manipulate asurgical tool. The control station is positioned remote from thearmature. The control station has a control handle configured to operatethe armature. The sensor is positioned to sense a vibration of thesurgical tool. The actuator is positioned to provide a vibration to thecontrol handle of the control station. The controller is incommunication with the sensor and the actuator. The controller isconfigured to receive data from the sensor corresponding to a sensedvibration. The controller is further configured to actuate the actuatorbased on the received data such that the actuator provides a vibrationto the control handle when the sensor senses the vibration of thesurgical tool.

In accordance with another aspect of the present invention, a method forproviding vibration feedback during robot-assisted surgery includesoperating an armature of a robotic surgical system using a controlhandle of a remotely positioned control station of the robotic surgicalsystem, sensing a vibration of a surgical tool with a sensor while thesurgical tool is coupled to the armature of the robotic surgical system,and actuating an actuator to provide a vibration to a control handle ofthe control station of the robotic surgical system when the sensorsenses the vibration of the tool.

In accordance with yet another aspect of the present invention, a systemfor configuring a robotic surgery system to provide vibration feedbackduring robot-assisted surgery includes a sensor, an actuator, and acontroller. The robotic surgery system includes an armature formanipulating a surgical tool and a control station having a controlhandle for operating the armature. The sensor is configured for couplingto the robotic surgery system such that the sensor senses a vibration ofthe surgical tool. The actuator is configured for coupling to thesurgery system such that the actuator provides a vibration to thecontrol handle. The controller is configured to be electrically coupledwith the sensor and the actuator. The controller is configured toreceive data corresponding to a sensed vibration from the sensor. Thecontroller is further configured to actuate the actuator based on thereceived data such that the actuator provides a vibration to the controlhandle when the sensor senses the vibration of the tool.

In accordance with still another aspect of the present invention, amethod for configuring a robotic surgery system to provide vibrationfeedback during robot-assisted surgery includes coupling a sensor to therobotic surgical system such that the sensor senses a vibration of asurgical tool, coupling an actuator to the surgical system such that theactuator provides a vibration to a control handle, and electricallycoupling a controller with the sensor and the actuator. The controlleris configured to receive data corresponding to a sensed vibration fromthe sensor. The controller is further configured to actuate the actuatorbased on the received data such that the actuator provides a vibrationto the control handle when the sensor senses the vibration of the tool.

In accordance with another aspect of the present invention, a roboticsurgery system includes an armature, a control station, a sensor, anactuator, and a controller. The armature is configured to manipulate asurgical tool. The control station is positioned remote from thearmature. The control station has a control handle configured to operatethe armature. The sensor is coupled to sense a signal such as a sound oran audio signal or a vibration generated by the surgical tool. Theactuator is coupled to provide an audio signal to an operator of thecontrol handle. The controller is in communication with the sensor andthe actuator. The controller is configured to receive data correspondingto a sensed signal from the sensor. The controller is further configuredto actuate the actuator based on the received data such that theactuator provides an audio signal to an operator of the control handlewhen the sensor senses the signal generated by the tool.

In accordance with yet another aspect of the present invention, arobotic system includes a robotic component, a control station, asensor, an actuator, and a controller. The control station is positionedremote from the robotic component. The control station has a controlhandle configured for operating the robotic component. The sensor iscoupled to sense a vibration of the robotic component. The actuator iscoupled to provide a vibration to the control handle. The controller isin communication with the sensor and the actuator. The controller isconfigured to receive data corresponding to a sensed vibration from thesensor. The controller is further configured to actuate the actuatorbased on the received data such that the actuator provides a vibrationto the control handle when the sensor senses the vibration of therobotic component.

In accordance with still another aspect of the present invention, amethod for performing robot-assisted surgery includes manipulating asurgical tool coupled to a robotic armature of a robotic surgery systemusing a control handle of a control station positioned remotely from therobotic armature, thereby generating a vibration of the surgical tool. Avibration is received at the control handle of the control station, thereceived vibration at the control handle corresponding to the generatedvibration of the surgical tool.

In accordance with yet another aspect of the present invention, arobotic surgery system includes an armature, a control station, asensor, an actuator, and a controller. The armature is configured formanipulating a surgical tool. The control station is positioned remotefrom the armature and has a control handle configured for operating thearmature. The sensor is positioned to sense a vibration of the surgicaltool. The actuator is positioned to provide a vibration at the controlstation. The controller is in communication with the sensor and theactuator. The controller is configured to receive data from the sensorcorresponding to a sensed vibration. The controller is furtherconfigured to actuate the actuator based on the received data such thatthe actuator provides a vibration at the control station.

In accordance with another aspect of the present invention, a method forproviding vibration feedback during robot-assisted surgery includesoperating an armature of a robotic surgical system from a remotelypositioned control station of a robotic surgical system, sensing avibration of a surgical tool with a sensor while the surgical tool iscoupled to the armature of the robotic surgical system, and actuating anactuator to provide a vibration at the control station of the roboticsurgical system when the sensor senses the vibration of the tool.

In accordance with still another aspect of the present invention, amethod for training an operator of a robotic surgery system includesenabling the operator to perform a test procedure, recording vibrationsof a surgical tool of the robotic surgery system during the testprocedure, and generating a score for the operator based at least inpart on the recorded vibrations of the surgical tool.

In accordance with yet another aspect of the present invention, arobotic surgery system includes an armature, a control station, asensor, and a controller. The armature is configured for manipulating asurgical tool. The control station is positioned remote from thearmature and has a control handle configured for operating the armature.The sensor is positioned to sense a vibration of the surgical tool. Thecontroller is in communication with the sensor. The controller isconfigured to record vibrations of the surgical tool during a testprocedure performed by an operator, and generate a score for theoperator based at least in part on the recorded vibrations of thesurgical tool.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is best understood from the following detailed descriptionwhen read in connection with the accompanying drawings. It is emphasizedthat, according to common practice, the various features of the drawingsare not to scale. On the contrary, the dimensions of the variousfeatures may be arbitrarily expanded or reduced for clarity. Included inthe drawings are the following figures:

FIG. 1 depicts an exemplary embodiment of a system for configuring arobotic surgery system to provide vibration feedback duringrobot-assisted surgery in accordance with aspects of the presentinvention;

FIG. 2A depicts an exemplary embodiment of a sensor assembly of thesystem of FIG. 1;

FIG. 2B depicts the sensor assembly shown in FIG. 2A mounted to arobotic surgery system;

FIG. 2C depicts an exemplary embodiment of a sensor mount component ofthe sensor assembly shown in FIG. 2A;

FIG. 3A depicts an exemplary embodiment of an actuator assembly of thesystem of FIG. 1;

FIG. 3B depicts the actuator assembly shown in FIG. 3A mounted to arobotic surgery system;

FIG. 3C depicts an exemplary embodiment of an actuator mount componentof the actuator assembly shown in FIG. 3A;

FIG. 4 is a flowchart depicting an exemplary method for configuring arobotic surgery system to provide vibration feedback duringrobot-assisted surgery in accordance with aspects of the presentinvention;

FIG. 5A depicts an exemplary embodiment of a robotic surgical system inaccordance with aspects of the present invention;

FIG. 5B depicts an exemplary embodiment of a robotic armature of thesystem of FIG. 5A;

FIG. 5C depicts an exemplary embodiment of a control handle of thesystem of FIG. 5A;

FIG. 5D depicts an exemplary embodiment of an actuator of the system ofFIG. 5A

FIG. 5E depicts an exemplary embodiment of an amplitude controller ofthe system of FIG. 5A;

FIG. 6 depicts another exemplary embodiment of a robotic armature andsurgical tool of the system of FIG. 5A;

FIG. 7 depicts an exemplary control station of the system of FIG. 5A;

FIG. 8 is a flowchart depicting an exemplary method for providingvibration feedback during robot-assisted surgery in accordance withaspects of the present invention;

FIG. 9 is a flowchart depicting an exemplary method for performingrobot-assisted surgery in accordance with aspects of the presentinvention;

FIG. 10 is a graph depicting an exemplary vibration sensed by a sensorand an exemplary vibration provided by an actuator in accordance withaspects of the present invention; and

FIG. 11.

DETAILED DESCRIPTION OF THE INVENTION

The exemplary systems and methods disclosed herein are suitable forintegration with robotic systems. For example, the system componentsdescribed herein may be integrated with a robotic system having aremotely controlled robotic component. Suitable robotic systems mayinclude a control station by which an operator may remotely operate therobotic component. The control station may include one or more controlhandles that are manipulated by the operator in operating the roboticcomponent. In this configuration, the motions of the control handles maybe transmitted to the robotic component, which will performcorresponding motions. An exemplary robotic system for use with thepresent invention is the DA VINCI Surgical System, provided by IntuitiveSurgical, Inc.

The exemplary systems and methods disclosed herein may be particularlyuseful for use in conjunction with robotic surgical systems. As will bedescribed below, the systems and methods disclosed herein may providesensory feedback such as vibration feedback during the performance of arobot-assisted surgery, i.e., a surgery employing a robotic surgerysystem. As used herein, the term “vibration feedback” is intended toencompass tactile feedback as well as audio feedback. The vibrationfeedback may augment the surgeon's sensory experience during operations,thereby reducing cognitive load, and enabling a surgeon to performrobot-assisted surgeries more precisely, more quickly, and/or withgreater ease and pleasure.

Although the systems and methods described below generally relate torobotic surgical systems, it is contemplated that aspects of the presentinvention may be used with non-surgical robotic systems withoutdeparting from the spirit and scope of the present invention. In otherwords, the present invention may be useful with medical robotic systemsother than surgical robotic systems or with non-medical robotic systems.

For example, tactile feedback can benefit any robotic system in which auser's sense of touch is dissociated from a robotic tool. This includes,for example, any system in which the user is remote from a robotic tool(i.e., a system in which the hands of the user are not in direct contactwith the tool). In the context of surgical robotic systems according toaspects of the invention, this encompasses any surgical system in whichthe hands of the surgeon are not in direct contact with the surgicaltool (e.g., endoscopic or other minimally invasive surgical devices)that manipulates the patient. In other words, any system could benefitwhere the user-master interface (e.g., the interface between the surgeonand the controls of a surgical robotic system) does not otherwiseapproximate the slave-subject interface (e.g., the interface between therobot's tools and the patient's body in a surgical robotic system) in atactile sense.

Specifically, doctors often use long thin tools to reach deep within apatient's body, which separates the doctors from the site of theintervention. In the case of a surgical robot system, these tools may bestiff rods, so the vibrations are transmitted to the robot arms holdingthe tools.

In other applications, the vibration sensor is optionally attached tothe tool itself, perhaps located deep in the body. Endoscopicprocedures, for example, use a lens on the end of a flexible instrument(an endoscope) to look into a patient's throat, and colonoscopiesexplore the bowels in a similar way. In both of these cases, the doctor(typically a gastroenterologist) is visually examining the tissue forsigns of irregularity. Suspicious patches are biopsied by sending a thinflexible tool down the working channel of the scope. Maneuvering thisbiopsy tool is quite challenging, and it may benefit the doctor to beable to receive tactile signals during the manipulation.

Venous access is another potential application for aspects of thisinvention, in which doctors pass thin catheters from outside the body upthrough a vein or an artery to get to the heart or other anatomicalstructures. Doctors may be operating based on low-quality images, suchas a 2D fluoro image, where anatomical structures are hard to see.Injecting contrast dye helps them to see better, but they have virtuallyno sense of touch. In such an application, a vibration sensor isoptionally placed at the tip of the catheter.

Beyond medicine, there are many cases when a human operator controls arobot that is in a distant and/or hazardous environment. For example,defusing improvised explosive devices in a war zone is one example ofsuch an application. Performing more sophisticated actions like openingdoors or searching disaster scenes will require better feedback for theoperator. In such instances, vibrotactile feedback would be beneficial.

Exploring the deep ocean or other hostile areas would benefit from thisinvention, especially if the robot needs to directly manipulate itsenvironment. Even just driving a wheeled robot around could be madeeasier if one could feel the type of terrain that it is traversing.

Traditional surgical skill assessment relies on observation of asurgeon's performance, a method that is both subjective and timeconsuming. The growing demand for robotic minimally invasive surgery hasincreased the need for objective methods of assessing technical skillfor surgical training. One possible method of objectively accounting forthe quality of these interactions and classifying instrument handlingskills is to measure the transient mechanical vibrations of the roboticinstruments. These vibrations primarily result from instrument contactwith objects in the environment, such as collisions and needle handoffs,with larger vibrations generally signifying rougher interactions. Abruptmovements of the surgical instruments also cause measurable vibrations.The inventors have demonstrated that robotic instrument vibrations caneasily be measured with low-cost accelerometers mounted on thepatient-side robot.

The inventors have demonstrated that instrument vibrations, as measuredby the disclosed embodiments, are a construct valid measure of technicalsurgical skill during robotic in vitro training tasks. Althoughdifferences in RMS vibration magnitude appear to be small for thesetasks, some novices' slower coordinated actions may result in lowervibrations for certain manipulation events. Accordingly, overall skilllevel may depend on multiple measures, including the abilities tocomplete tasks quickly, efficiently, with low forces, and with lowinstrument vibrations.

The exemplary systems and methods disclosed herein may also beparticularly useful for training users of robotic surgery systems. Aswill be described below, the systems and methods disclosed herein mayanalyze the vibrations occurring during use of the robotic surgerysystem, and generate a score for the user of the robotic surgery systembased on the analyzed vibrations. As used herein, the term “score”refers to any grade, level, rank, amount, total, or other value, whethernumerical, graphical, oral, or textual, representing the aptitude of theuser of the robotic surgery system. The score provided by the roboticsurgery system may be relied on or considered by the user to attempt toimprove their performance with the robotic surgery system (e.g., throughrepeated uses and monitoring of a trend in the user's score). The scoremay assist the operator in improving their performance by encouraginglower magnitudes of vibrations, fewer instances of vibration, or fastertimes for performing the procedure.

Referring now to the drawings, FIG. 1 illustrates an exemplary system100 for configuring a robotic surgery system to provide vibrationfeedback during robot-assisted surgery in accordance with an aspect ofthe present invention. System 100 may configure the robotic surgerysystem to provide tactile feedback and/or audio feedback to a user.Suitable robotic surgery systems usable with system 100 may include anarmature for manipulating a surgical tool and a control station having acontrol handle for operating the armature. As an overview, system 100includes a sensor 102, actuators 104, controller 106, and amplitudecontroller 108. Additional details of system 100 are described below.

Sensor 102 is configured to be coupled to a robotic surgery system. Inan exemplary embodiment, sensor 102 is configured to be coupled to arobotic surgery system in a location where sensor 102 can sense avibration of a surgical tool or a sound made by the surgical tool.Sensor 102 may be configured to be mounted directly to an armature ofthe surgical system near the base of the surgical tool. It may bedesirable to mount sensor 102 to the system armature, rather thandirectly to the surgical tool being manipulated, in order to avoidremounting sensor 102 whenever a surgical tool is changed, and becausehigh-frequency vibrations and sounds (like those sensed by sensor 102)transmit well through solid objects. Additionally, sensor 102 may beconfigured to be mounted in an area of the robotic surgery system thatis outside of a sterile area. A robotic surgery system may have asterile area corresponding to an area in which an operation will beperformed on a patient. It may be desirable to mount sensor 102 outsideof this sterile area in order to avoid having to sterilize sensor 102.

While two sensors 102 are illustrated, it will be understood that system100 may include any number of sensors 102. For example, system 100 mayinclude one sensor for each armature or surgical tool employed by therobotic surgery system. Additionally, system 100 may include multiplesensors for each armature or surgical tool employed by the roboticsurgery system.

Sensor 102 is configured to sense a vibration of the surgical tool.During a surgical procedure, high-frequency vibrations may naturallyoccur during hard contact, cutting, rubbing, puncture, and a host ofother actions with the surgical tool. The vibrations sensed by sensor102 may desirably be these high frequency vibrations, between about 10Hz and 1000 Hz. The vibrations sensed by sensor 102 may also includeaudible sounds in this frequency range. The system is optionallyreconfigurable so that one can change the bandwidth to suit a particularapplication or a user's preference. For example, the passband could beadjusted to start below or above 10 Hz, end below or above 1000 Hz, andoptionally remove intermediate frequency ranges.

In an exemplary embodiment, sensor 102 is an accelerometer. Theaccelerometer may include multiple measurement axes for measuring thevibration of the surgical tool in multiple dimensions. A suitableaccelerometer for use as sensor 102 includes, for example, a MEMS-basedhigh-bandwidth accelerometer, capacitive accelerometers, piezoelectricor piezoresistive accelerometers, Hall effect accelerometers,magnetoresistive accelerometers, or heat transfer accelerometers, orother suitable accelerometers. In one embodiment, for example, ADXL322chips provided by Analog Devices, Inc. are optionally used.Alternatively, sensor 102 may be the LIS344ALH three-axis linearaccelerometer, provided by STMicroelectronics. Other suitableaccelerometers will be known to one of ordinary skill in the art fromthe description herein.

In another exemplary embodiment, sensor 102 is a sound or noise sensor,e.g., a microphone. The microphone may be configured to record anysounds made by the surgical tool during the surgery. Suitablemicrophones for use as sensor 102 will be known to one of ordinary skillin the art from the description herein.

Sensor 102 may be coupled to an armature of the robotic surgery systemusing a sensor assembly. FIG. 2A illustrates an exemplary sensorassembly in accordance with aspects of the present invention. In anexemplary embodiment, the sensor assembly includes sensor 102 and asensor mount 103. FIG. 2B illustrates the exemplary sensor assemblymounted to a robotic armature of a robotic surgery system. Sensor mount103 may affix sensor 102 to the robotic surgery system such that sensor102 directly contacts an armature 110 of the surgery system. It may bedesirable for sensor 102 to directly and/or rigidly contact the armature110 in order to increase the transmission of vibrations from thesurgical tool to sensor 102.

FIG. 2C illustrates an exemplary sensor mount of the sensor assembly. Inan exemplary embodiment, a sensor mount 103 includes a recess 130 forreceiving sensor 102. Where sensor 102 uses a wire to transmit data,sensor mount 103 may optionally include a recess 132 and/or an opening134 for the wire to pass through and attach to sensor 102. Sensor mount103 further includes engagement surfaces 136 for engaging a roboticarmature 110 of the robotic surgery system. Sensor mount 103 may furtherinclude flanges 138 for securing sensor mount 103 in place. Sensor mount103 may couple sensor 102 to the robotic surgery system via a frictionfit between engagement surfaces 136. Alternatively, sensor mount 103 mayuse snaps, bolts, straps, hook and loop fasteners such as VELCRO, oradhesives to couple sensor 102 to the robotic surgery system. Sensormount 103 may desirably be adjustable such that sensor 102 can bemounted to multiple different types of robotic surgery systems or inmultiple different locations of a robotic surgery system. Suitablematerials for sensor mount 103 include, for example, acrylonitrilebutadiene styrene (ABS) plastic. Other suitable materials for sensormount 103 will be known to one of ordinary skill in the art from thedescription herein.

Sensor 102 is configured to transmit data corresponding to a sensedvibration. Desirably, sensor 102 solely transmits data corresponding tovibrations of the surgical tool or sounds made by the surgical tool.Sensor 102 may transmit the data via a wire, as shown in FIG. 1, orwirelessly, as would be understood by one of ordinary skill in the art.

As described above, sensor 102 or a supplemental sensor may sense audiosignals generated by the surgical tool. Sensor 102 may be configured todetect both vibrations of the surgical tool and sounds made by thesurgical tool, as both may be characterized as high frequencyvibrations. Alternatively, sensor 102 may exclusively sense audiosignals (as opposed to other high frequency vibrations) generated by thesurgical tool. For example, sensor 102 may sense audio signals generatedfrom the surgical tool contacting the patient during an operation. Theaudio signals sensed by sensor 102 may desirably be between about 15 Hzand 20,000 Hz. Sensor 102 may be configured to transmit datacorresponding to the sensed audio signals.

Generally, the same vibrations that one can feel may also cause pressurewaves in the air, which one could perceive as sounds. Accordingly, aseparate microphone or other audio sensor may optionally be used inaddition to or instead of a vibrational sensor. Because the audio andthe accelerations appear similar on an oscilloscope (perhaps scaled by afactor), it is expected that no separate conversion would be needed toconvert the data from a vibrational sensor from a high frequencyvibration to an audio signal.

Actuators 104 are configured to be coupled to a robotic surgery system.In an exemplary embodiment, actuators 104 are configured to be coupledto a robotic surgery system in a location where actuators 104 canprovide a vibration to a control handle. A robotic surgery system mayinclude a control handle to be manipulated by an operator in operatingthe armature or surgical tool. Actuators 104 may be mounted directly onthe control handle. Alternatively, actuators 104 may be mounted on aseparate structure that is in physical contact with the control handle.It may be desirable to mount actuators 104 on a portion of the controlhandle removed from the area held by the operator of the control handle,in order to avoid interfering with the manipulation of the controlhandle by the operator.

When actuators 104 are mounted to the control handles of an existingrobotic surgery system, it may be necessary to affix a counter-balanceto the control handles to account for the weight of the actuators.Determination of a suitable weight and location for the counter-balancewill be understood by one of ordinary skill in the art. Alternatively, acounter-balance to actuators 104 may be implemented by software in therobotic surgery system.

While two actuators 104 are illustrated, it will be understood thatsystem 100 may include any number of actuators 104. For example, system100 may include one actuator for each armature or surgical tool employedby the robotic surgery system. Additionally, system 100 may include anactuator 104 for each sensor 102. Finally, system 100 may include one ormore actuators 104 for each control handle at the control station of therobotic surgical system. Multiple actuators may be employed on thecontrol handles to enable system 100 to provide vibrations in multipledifferent directions, or provide larger vibrations. This may enhance therealism of the vibrations felt by the operator of the robotic surgerysystem.

Actuators 104 are configured to provide a vibration to the controlhandle. The vibrations provided by actuators 104 may desirably be highfrequency vibrations, between about 10 Hz and 1000 Hz. Further, thevibrations provided by actuators 104 may match the frequency of thevibrations of the surgical tool sensed by sensor 102. FIG. 10illustrates a graph of an exemplary vibration sensed by a sensor and acorresponding exemplary vibration provided by an actuator in accordancewith an aspect of the present invention. It may be desirable to matchthe frequencies of the vibrations in order to accurately reproduce atthe control handles the feeling of handling the surgical tool during theprocedure.

Further, actuators 104 may not provide vibrations to the control handlesat certain predetermined frequencies. For example, the robotic armaturemay have one or more resonant frequencies, or one or more naturalvibrating frequencies based on operation of the motors that move thearmature. Providing vibrations to the control handles at this frequencymay impart a vibrating movement to the armatures, and thereby createundesirable vibrational feedback. Thus, actuators 104 may be configurednot to provide vibrations at the resonant or natural vibratingfrequencies of the armature in order to avoid generating unwantedfeedback.

In an exemplary embodiment, actuators 104 are voice coil actuators.Suitable voice coil actuators for use as actuators 104 include, forexample, voice coil linear actuators provided by H2W Technologies, Inc.or BEI Kimco Magnetics. Other suitable actuators will be known to one ofordinary skill in the art from the description herein.

Actuators 104 may be coupled to the control station of the roboticsurgery system using an actuator assembly. FIG. 3A illustrates anexemplary actuator assembly in accordance with aspects of the presentinvention. In an exemplary embodiment, the actuator assembly includesactuator 104 and an actuator mount 105. FIG. 3B illustrates theexemplary actuator assembly mounted to a control handle of a roboticsurgery system. In an exemplary embodiment, actuator 104 is a voice coilactuator having a wire coil 140 and a permanent magnet (not shown).Actuator mount 105 may mount the actuator 104 such that the wire coil140 is rigidly attached to the control handle 116 and the permanentmagnet is mounted on a linear bearing 142, and is centered by springs.The permanent magnet may therefore be free to move back and forth onlinear bearing 142 in response to the current through the wire coil 140.Alternatively, actuator mount 105 may be configured to hold thepermanent magnet stationary, and allow the wire coil to move on a linearbearing. Thus, actuator mount 105 may allow actuator 104 to provide avibration from either the permanent magnet or the wire coil to thecontrol handle 116.

FIG. 3C illustrates an exemplary actuator mount of the actuatorassembly. In an exemplary embodiment, an actuator mount 105 includes acavity 150 for receiving actuator 104. Actuator mount 105 furtherincludes a mounting surface 152 having mounting holes 154, for rigidlysecuring the actuator 104 to the actuator mount 105. Actuator mount 105further includes engagement surfaces 156 for engaging a control handle116 of the robotic surgery system. Actuator mount 105 may furtherinclude an opening 158 for receiving a linear bearing 142 of theactuator 104. Actuator mount 105 may couple actuator 104 to the roboticsurgery system via a friction fit between engagement surfaces 156.Alternatively, actuator mount 105 may use snaps, bolts, straps, hook andloop fasteners such as VELCRO, or adhesives to couple actuator 104 tothe robotic surgery system. Actuator mount 105 may desirably beadjustable such that actuator 104 can be mounted to multiple differenttypes of robotic surgery systems or in multiple different locations of arobotic surgery system. Suitable materials for actuator mount 105include, for example, acrylonitrile butadiene styrene (ABS) plastic.Other suitable materials for actuator mount 105 will be known to one ofordinary skill in the art from the description herein.

As described above, actuators 104 may be mounted on a separate structurethat is in physical contact with the control handle. As illustrated inFIG. 3B, control handle 116 may multiple fixed-length arm portions 118and multiple joints 120, to enable control handles 116 to be movable inall three dimensions and angles of freedom. It may be desirable foractuator mount 105 to mount actuator 104 to an arm portion 118 removedfrom a portion of the control handle 122 grasped by an operator of therobotic surgery system, in order to avoid interfering with themanipulation of the control handle by the operator.

While actuators 104 are described above as providing vibrations to thecontrol handle, it will be understood that actuators 104 may alsoprovide audio signals to the operator of the control handle.Alternatively, actuators 104 may exclusively provide audio signals (asopposed to other high frequency vibrations) to an operator of thecontrol handle. For example, actuators 104 may be speakers, and may bemounted at the control station to provide audio signals matching theaudio signals generated by the surgical tool during an operation. Theaudio signals provided by actuators 104 may desirably be maintainedbetween about 15 Hz and 20,000 Hz.

In one exemplary embodiment, actuators 104 are speakers. In thisembodiment, the speakers are configured to be coupled to a roboticsurgery system in a location where they can provide sound to a user ofthe robotic surgery system. For example, a robotic surgery system mayinclude a control station, at which the speakers are mounted. It will beunderstood that system 100 may include any number of speakers 104.Utilizing at least two speakers may be desirable in order to providestereo sound to the user of the robotic surgery system. Additionally, itwill be understood that system 100 may include a combination ofactuators 104 configured as speakers to provide audio feedback andactuators 104 configured to provide vibrations as tactile feedback.

The sounds provided by the speakers may desirably span the entireaudible range. Further, the sounds provided by the speakers desirablymatch the sounds caused by the surgical tool sensed by sensor 102. Itmay be desirable to match the sounds in order to accurately reproduce atthe control handles the feeling or experience of handling the surgicaltool during the procedure.

In this exemplary embodiment, actuators 104 are conventional stereocomputer speakers. Suitable speakers for use as actuators 104 will beknown to one of ordinary skill in the art from the description herein.

Controller 106 is configured to be electrically connected with sensor102 and actuators 104. Controller 106 is configured to receive data fromsensor 102. In an exemplary embodiment, controller 106 is configured toreceive data from sensor 102 corresponding to a sensed vibration of thesurgical tool. Controller 106 may then process the data from sensor 102.Then, controller 106 is configured to actuate actuators 104 based on thedata received from sensor 102. In an exemplary embodiment, controller106 actuates actuators 104 such that actuators 104 provide a vibrationto the control handle when sensor 102 senses a vibration of the surgicaltool.

Controller 106 determines the vibrations provided by actuators 104. Asdescribed above, the vibrations provided by actuators 104 may desirablybe high frequency vibrations, between about 10 Hz and 1000 Hz.Accordingly, controller 106 may only actuate actuators 104 when sensor102 senses vibration occurring within the selected frequency range ofvibrations. Further, controller 106 may not actuate actuators 104 atcertain predetermined frequencies in order to avoid generating unwantedfeedback, as described above.

Controller 106 may desirably amplify the vibrations provided to thecontrol handle from actuators 104 with respect to the vibrations sensedby sensors 102. Amplifying the vibrations may provide an operator of therobotic surgery system with a superior perception of the movement of thesurgical tool.

In an exemplary embodiment, controller 106 is a microcontroller.Controller 106 may have a number of data inputs and outputscorresponding to the number of sensors 102 and actuators 104,respectively. Suitable microcontrollers for use as controller 106 willbe known to one of ordinary skill in the art from the descriptionherein. Alternatively, controller 106 may comprise an analog circuit, aswould be understood by one of ordinary skill in the art.

Controller 106 is configured to transmit data for actuating actuators104. Controller 106 may transmit the data via a wire or wirelessly, aswould be understood by one of ordinary skill in the art. Desirably,controller 106 solely transmits data for actuating actuators 104 whensensor 102 senses a vibration of the surgical tool. Controller 106 mayfurther transmit data such that actuators 104 provide vibrations to thecontrol handle in real time corresponding to sensed vibrations of thesurgical tool by sensor 102. It may be desirable to actuate actuators104 in real time to improve the precision and response time for theoperator of the robotic surgery system.

While controller 106 is described above as actuating actuators 104 toprovide vibrations, it will be understood that controller 106 may alsoactuate actuators 104 to provide an audio signal to the operator of thecontrol handle based on the received data from sensor 102.Alternatively, controller 106 may exclusively actuate actuators 104 toprovide an audio signal (as opposed to other high frequency vibrations)to the operator of the control handle.

Controller 106 may desirably perform additional signal processing steps,as set forth below, in order to provide improved vibration feedback to auser of the robotic surgery system. While the steps set forth below aredescribed as being performed by controller 106, it will be understoodthat the steps may also be performed by associated processing componentsat sensors 102 or actuators 104.

As set forth above, sensor 102 may be an accelerometer configured tomeasure acceleration data in three different axes. Each axis ofacceleration data may be filtered with a first-order analog low-passfilter, with a cutoff frequency of 1000 Hz for example, in order tomatch the bandwidth of human vibration detection. Additionally, a firstorder high-pass filter with a cutoff frequency of 80 Hz for example maybe used to remove any DC component from each signal, and to reduce theoverall response at frequencies that may generate instability.

Controller 106 may also desirably sum the three acceleration signalsfrom sensor 102 in order to obtain one signal representative ofvibrations in all directions. Because the human hand is not sensitive tovibration direction, summing multiple acceleration signals withcontroller 106 adequately preserves their temporal and spectralfeatures. The summed signals may further be amplified to increase thesignal-to-noise ratio during the transmission of the accelerationsignals from sensors 102 to controller 106 or from controller 106 toactuators 104.

As set forth above, sensor 102 may also be an audio sensor, e.g., amicrophone. Controller 106 may desirably transmit audio signals directlyfrom sensors 102 to actuators 104 with additional processing.

System 100 may also include an amplitude controller 108. Amplitudecontroller 108 controls the amplitude of the vibrations provided to thecontrol handle by actuators 104. Alternatively, where actuators 104 arespeakers, amplitude controller 108 controls the volume of the soundsprovided at the control station by actuators 104. Where both vibrationaland auditory actuators 104 are used, it may be desirable to includeseparate amplitude controllers 108 for controlling the separatecomponents. In an exemplary embodiment, amplitude controller 108 ispositioned at the control station. Amplitude controller 108 may includea means for allowing an operator of the control station to adjust theamplitude of the vibrations or volume of sound provided by actuators104. For example, amplitude controller 108 may include a knob, dial,buttons, or other data input components for setting a desired amplitude.Amplitude controller 108 may be in communication with controller 106.Amplitude controller 108 may transmit data to controller 106 based onthe amplitude data input by the operator of the control station.Controller 106 may then modify the amplitude of the vibrations or volumeof sound provided by actuators 104 based on the data received fromamplitude controller 108. Amplitude controller 108 may control theamplitude of vibrations provided by actuators 104 such that the providedvibrations are ratiometric to the sensed vibrations. Similarly,amplitude controller 108 may control the volume of sounds provided byactuators 104 such that the provided sounds are ratiometric to thesensed sounds.

FIG. 4 is a flowchart illustrating an exemplary method 200 forconfiguring a robotic surgery system to provide vibration feedbackduring robot-assisted surgery in accordance with an aspect of thepresent invention. Method 200 may configure the robotic surgery systemto provide tactile feedback and/or audio feedback to a user. Method 200may be employed on robotic surgery systems including an armature formanipulating a surgical tool and a control station having a controlhandle for operating the armature. As an overview, method 200 includescoupling a sensor to a surgical system, coupling an actuator to asurgical system, and electrically connecting a controller with thesensor and the actuator. For the purposes of illustration, the steps ofmethod 200 are described herein with respect to the components of system100. Additional details of method 200 are described below.

In step 202, a sensor is coupled to the surgical system such that thesensor senses a vibration of the surgical tool. In an exemplaryembodiment, sensor 102 is coupled to a robotic surgery system in alocation where sensor 102 can sense a vibration of a surgical tool. Asdescribed above, sensor 102 may be mounted directly to an armature ofthe robotic surgery system. Sensor 102 may be mounted in an area of therobotic surgery system that is outside of a sterile area.

As described above, sensor 102 may be coupled to an armature of therobotic surgery system using a sensor mount. The sensor mount may besubstantially as described above in connection with system 100.

As described above, sensor 102 is configured to transmit datacorresponding to a sensed vibration. Desirably, sensor 102 solelytransmits data corresponding to vibrations of the surgical tool. Sensor102 may transmit the data via a wire or wirelessly, as would beunderstood by one of ordinary skill in the art.

As described above, while sensor 102 is described as sensing vibrationsof the surgical tool, it will be understood that sensor 102 may alsosense audio signals generated by the surgical tool. For example, sensor102 may be a microphone coupled to the surgical system in order to senseaudio signals generated from the surgical tool contacting the patientduring an operation. Sensor 102 may be configured to transmit datacorresponding to the sensed audio signals.

In step 204, an actuator is coupled to the surgical system such that theactuator provides a vibration to the control handle. In an exemplaryembodiment, actuator 104 is coupled to a robotic surgery system in alocation where actuator 104 can provide a vibration to the controlhandle. As described above, actuator 104 may be mounted directly on thecontrol handle. Actuator 104 may be mounted on a separate structure thatis in physical contact with the control handle.

As described above, actuator 104 may be coupled to the control stationof the robotic surgery system using an actuator mount. The actuatormount may be substantially as described above with respect to system100.

As described above, actuator 104 is configured to provide vibrations tothe control handle that correspond to a sensed vibration. Desirably, thevibrations provided by actuator 104 may be high frequency vibrations,between about 10 Hz and 1000 Hz. Further, the vibrations provided byactuators 104 may match the frequency of the vibrations of the surgicaltool sensed by sensor 102.

As described above, while actuators 104 are described as providingvibrations to the control handle, it will be understood that actuators104 may also provide audio signals to the operator of the controlhandle. For example, actuators 104 may be speakers, and may be mountedat the control station to provide audio signals matching the audiosignals generated by the surgical tool during an operation.

In step 206, a controller is electrically connected with the sensor andthe actuator such that the controller receives data from the sensor andactuates the actuator. In an exemplary embodiment, controller 106 iselectrically connected to sensor 102 to receive data from sensor 102corresponding to a sensed vibration of the surgical tool. Further,controller 106 is electrically connected to actuators 104 to actuate theactuators 104 based on the data received from sensor 102. Controller 106may actuate actuators 104 such that actuators 104 provide a vibration tothe control handle when sensor 102 senses a vibration of the surgicaltool. Controller 106 may only actuate actuators 104 when sensor 102senses vibration occurring within a selected frequency range ofvibrations. Controller 106 may further amplify the vibrations providedto the control handle from actuators 104 with respect to the vibrationssensed by sensors 102.

As described above, controller 106 is configured to transmit data foractuating actuators 104. Desirably, controller 106 solely transmits datafor actuating actuators 104 when sensor 102 senses a vibration of thesurgical tool. Controller 106 may transmit the data via a wire orwirelessly, as would be understood by one of ordinary skill in the art.

As described above, while controller 106 is described as actuatingactuators 104 to provide vibrations, it will be understood thatcontroller 106 may also actuate actuators 104 to provide an audio signalto the operator of the control handle based on the received data fromsensor 102.

Method 200 may also include the step of electrically connecting anamplitude controller with the controller. In an exemplary embodiment,amplitude controller 108 is electrically connected with controller 106.Amplitude controller 108 controls the amplitude of the vibration or thevolume of sound provided by the actuator 104. For example, amplitudecontroller 108 may control the amplitude of vibrations provided byactuators 104 such that the provided vibrations are ratiometric to thesensed vibrations.

System 100 is usable for configuring a robotic surgery system to providetactile feedback during robot-assisted surgery. It will be understood byone of ordinary skill in the art, however, that one or more of thecomponents of system 100 may integrated directly with a robotic surgerysystem. Accordingly, a robotic surgery system will now be described inaccordance with aspects of the present invention.

FIG. 5A illustrates an exemplary robotic surgery system 300 inaccordance with an aspect of the present invention. Robotic surgerysystem 300 may provide tactile feedback and/or audio feedback to a user.As an overview, system 300 includes sensors 302, actuators 304,controller 306, amplitude controller 308, robotic armatures 310,surgical tools 312, control station 314, and control handles 316.Additional details of system 300 are described below.

Robotic armatures 310 are for manipulating surgical tools 312. FIGS. 5Band 6 illustrate exemplary robotic armatures and surgical tools inaccordance with aspects of the present invention. In an exemplaryembodiment, armatures 310 are configured to receive surgical tools 312.Surgical tools 312 may be any tools usable during a surgical procedure.Suitable surgical tools 312 will be known to one of ordinary skill inthe art. Surgical tools 312 may be mounted to one or more armatures 310.

As illustrated in FIG. 6, robotic armatures 310 may include multiplefixed-length arm portion and multiple joints. Each of the joints mayinclude actuators for bending adjacent fixed-length arm portionsrelative to each other. This may enable the end of each armature 310 tobe movable in all three dimensions and angles of freedom. Thus, roboticarmatures 310 may manipulate surgical tools 312 in all three dimensionsand angles of freedom. This freedom of motion may be desirable forperforming surgical procedures without limitation. Suitable roboticarmatures 310 include the armatures of the DA VINCI Surgical System,provided by Intuitive Surgical, Inc. Other suitable robotic armatures310 will be known to one of ordinary skill in the art from thedescription herein.

While two robotic armatures 310 are illustrated, it will be understoodthat system 300 may include any number of armatures 310. For example,system 300 may include an armature 310 for each surgical tool 312employed by the robotic surgery system 300. Alternatively, system 300may include multiple surgical tools 312 on one armature 310.

Control station 314 is for operating robotic armatures 310. FIG. 7illustrates an exemplary control station in accordance with aspects ofthe present invention. In an exemplary embodiment, control station 314includes control handles 316 for operating robotic armatures 310.Control station 314 allows an operator, e.g., a surgeon, to perform arobot-assisted surgery by controlling a surgical tool 312 held byrobotic armatures 310. To this end, control station 314 may beelectrically connected with robotic armatures 310.

Thus, control station 314 may transmit data corresponding to the motionsof control handles 316 to the robotic armatures 310. Robotic armatures310 may then move in motions corresponding to the motions of controlhandles 316, e.g., by actuating the actuators in the arm joints of thearmatures 310. Control station may further include visual and/or audiofeedback for indicating to the operator what is occurring at thelocation of the surgery.

FIG. 5C further illustrates exemplary control handles in accordance withaspects of the present invention. Each control handles 316 may includemultiple fixed-length arm portions 318 and multiple joints 320, toenable the control handle 316 to be movable in all three dimensions andangles of freedom, similarly to armatures 310. Alternatively, controlstation 314 may include one or more control handles 316, each controlhandle operable to move a robotic armature 310 in only a single degreeof freedom. Suitable control stations and control handles include thecontrol station and control handles of the DA VINCI Surgical System,made by Intuitive Surgical, Inc. Other suitable control stations 314 andcontrol handles 316 will be known to one of ordinary skill in the artfrom the description herein.

While two control handles 316 are illustrated, it will be understoodthat system 300 may include any number of control handles 316. Forexample, system 300 may include a control handle 316 for each roboticarmature 310.

Sensors 302 are coupled to robotic surgery system 300. In an exemplaryembodiment, sensors 302 are coupled to system 300 in a location wheresensor 302 can sense a vibration of surgical tool 312. Sensors 302 maybe integrated as parts of armatures 310 or surgical tool 312. It may bedesirable to integrate sensors 302 into armatures 310 in order to limitthe number of sensors 302 if one armature 310 may be used for multiplesurgical tools 312. Additionally, it may be desirable to incorporatesensors 302 in armatures 310 so that sensors 302 may utilize the wiringand power already provided for armatures 310. Alternatively, sensors 302may be mounted to armatures 310 near the base of surgical tools 312, asdescribed above with respect to system 100. Sensors 302 may be mountedto armatures 310 using sensor mounts 303, which may be sensor mountssubstantially as described above with respect to sensor mount 103.

As described above with respect to system 100, it will be understoodthat system 300 may include any number of sensors 302. In an exemplaryembodiment, system 300 may include one sensor 302 for each roboticarmature 310 employed by the robotic surgery system 300.

As described above with respect to system 100, sensors 302 areconfigured to sense a vibration of surgical tool 312. The vibrationssensed by sensors 302 may desirably be high frequency vibrations,between about 10 Hz and 1000 Hz. In an exemplary embodiment, sensors 302are accelerometers. Suitable accelerometers for use as sensors 302include any of the accelerometers described above with reference tosensor 102. Other suitable accelerometers will be known to one ofordinary skill in the art from the description herein.

Sensors 302 are configured to transmit data corresponding to a sensedvibration. Desirably, sensors 302 solely transmit data corresponding tovibrations of surgical tool 312. Sensors 302 may transmit the data via awire or wirelessly, as would be understood by one of ordinary skill inthe art.

As described above with respect to system 100, it will be understoodthat sensors 302 may also sense audio signals generated by surgical tool312. Alternatively, sensors 302 may exclusively sense audio signals (asopposed to other high frequency vibrations) generated by surgical tool312. For example, sensors 302 may sense audio signals generated fromsurgical tool 312 contacting the patient during an operation. Sensors302 may be configured to transmit data corresponding to the sensed audiosignals.

Actuators 304 are coupled to robotic surgery system 300. In an exemplaryembodiment, actuators 304 are coupled to system 300 in a location whereactuators 304 can provide a vibration to control handles 316, as shownin FIG. 5C. Actuators 304 may be integrated as parts of control handles316. Where control handles 316 have more than one fixed-length armportion connected by one or more joints (as described above), actuators304 may be disposed on a fixed-length arm portion removed from theportion of control handles 316 held by the operator. It may be desirableto incorporate sensors 302 in armatures 310 so that sensors 302 mayutilize the wiring and power already provided for armatures 310.Alternatively, actuators 304 may be mounted to control station 314 orcontrol hands 316, as described above with respect to system 100.Actuators 304 may be mounted to control handles 316 using actuatormounts 305, which may be actuator mounts substantially as describedabove with respect to actuator mount 105.

As described above with respect to system 100, it will be understoodthat system 300 may include any number of actuators 304. In an exemplaryembodiment, system 300 may include one actuator 304 for each controlhandle 316 of control station 314.

Actuators 304 are configured to provide a vibration to control handle316. The vibrations provided by actuators 304 may desirably be highfrequency vibrations, between about 10 Hz and 1000 Hz. Further, thevibrations provided by actuators 304 may match the frequency of thevibrations of surgical tool 312 sensed by sensors 302.

Further, actuators 304 may not provide vibrations to control handles 316at certain predetermined frequencies. For example, robotic armatures 310may have one or more resonant frequencies, or one or more naturalvibrating frequencies based on operation of the joints that move thearmatures 310. Providing vibrations to control handles 316 at thesefrequencies may impart a vibrating movement to the armatures 310, andthereby create undesirable vibrational feedback. Thus, actuators 304 maybe configured not to provide vibrations at the resonant or naturalvibrating frequencies of the armature 310 in order to avoid generatingunwanted feedback.

In an exemplary embodiment, actuators 304 are voice coil actuators.Suitable voice coil actuators for use as actuators 304 include any ofthe voice coil actuators described above with reference to actuators104. Other suitable actuators will be known to one of ordinary skill inthe art from the description herein.

As described above with respect to system 100, it will be understoodthat actuators 304 may also provide audio signals to the operator ofcontrol handles 316. Alternatively, actuators 304 may exclusivelyprovide audio signals (as opposed to other high frequency vibrations) toan operator of control handles 316. For example, actuators 304 may bespeakers, and may be mounted at control station 316 to provide audiosignals matching the audio signals generated by surgical tool 312 duringan operation, as shown in FIG. 5D.

Controller 306 is in communication with sensors 302 and actuators 304.Controller 306 is configured to receive data from sensors 302. In anexemplary embodiment, controller 306 is configured to receive data fromsensors 302 corresponding to a sensed vibration of a surgical tool 312.Further, controller 306 is configured to actuate actuators 304 based onthe data received from sensors 302. In an exemplary embodiment,controller 306 actuates actuators 304 such that actuators 304 provide avibration to control handles 316 when sensor 302 senses a vibration ofsurgical tool 312.

Controller 306 determines the vibrations provided by actuators 304. Asdescribed above, the vibrations provided by actuators 304 may desirablybe high frequency vibrations, between about 10 Hz and 1000 Hz.Accordingly, controller 306 may only actuate actuators 304 when sensor302 senses vibration of surgical tool 312 occurring within the selectedfrequency range of vibrations. Further, controller 306 may not actuateactuators 304 at certain predetermined frequencies in order to avoidgenerating unwanted feedback, as described above. Controller 306 mayfurther amplify the vibrations provided to control handles 316 fromactuators 304 with respect to the vibrations sensed by sensors 302.

In an exemplary embodiment, controller 306 is a microcontroller.Controller 306 may have a number of data inputs and outputscorresponding to the number of sensors 302 and actuators 304,respectively. Controller 306 may be integrated with a master controller(not shown) for the robotic surgical system 300. In this embodiment,controller 306 may utilizing the wiring and power already in place inthe robotic surgery system 300. Alternatively, controller 306 may be aseparate controller, as described above with respect to system 100.Suitable microcontrollers for use as controller 306 will be known to oneof ordinary skill in the art from the description herein.

Controller 306 is configured to transmit data for actuating actuators304. Controller 306 may transmit the data via a wire or wirelessly, aswould be understood by one of ordinary skill in the art. Desirably,controller 306 solely transmits data for actuating actuators 304 whensensors 302 sense a vibration of surgical tool 312. Controller 306 mayfurther transmit data such that actuators 304 provide vibrations tocontrol handles 316 in real time corresponding to sensed vibrations ofsurgical tool 312 by sensors 302. It may be desirable to actuateactuators 304 in real time to improve the precision and response timefor the operator of the robotic surgery system 300.

As described above with respect to system 100, it will be understoodthat controller 306 may also actuate actuators 304 to provide an audiosignal to the operator of control handles 316 based on the received datafrom sensors 302. Alternatively, controller 306 may exclusively actuateactuators 304 to provide an audio signal (as opposed to other highfrequency vibrations) to the operator of control handles 316.

System 300 may also include an amplitude controller 308. FIG. 5Eillustrates an exemplary amplitude controller in accordance with aspectsof the present invention. Amplitude controller 308 controls theamplitude of the vibrations provided to control handles 316 by actuators304. In an exemplary embodiment, amplitude controller 308 is positionedat control station 314. Amplitude controller 308 may be an amplitudecontroller substantially as described above with respect to amplitudecontroller 108.

Robotic surgery system 300 described above may also be usable fortraining operators to improve their performance in performingrobot-assisted surgery. When so used, controller 306 may be configuredto record the vibrations sensed by sensor 302, and to generate a scorefor the operator based on the recorded vibrations. Additional detailsregarding this application for robotic surgery system 300 are describedin greater detail herein.

FIG. 8 is a flowchart illustrating an exemplary method 400 for providingvibration feedback during robot-assisted surgery in accordance with anaspect of the present invention. Method 400 may provide tactile feedbackand/or audio feedback to a user during the robot-assisted surgery. As anoverview, method 400 includes sensing a vibration of a surgical tool andactuating an actuator such that the actuator provides a vibration to acontrol handle. For the purposes of illustration, the steps of method400 are described herein with respect to the components of system 300.Additional details of method 400 are described below.

Method 400 may be performed by a robotic surgery system. In an exemplaryembodiment, a robotic surgery system 300 is provided. Robotic surgerysystem 300 includes at least one robotic armature 310 for manipulating asurgical tool 312 and a control station 314 having a control handle 316for operating the robotic armature 310. A suitable robotic surgerysystem 300 is the DA VINCI Surgical System, made by Intuitive Surgical,Inc. Other suitable robotic surgery systems will be known to one ofordinary skill in the art from the description herein.

In step 402, a vibration of the surgical tool is sensed with a sensor.In an exemplary embodiment, sensor 302 senses a vibration of surgicaltool 312. Sensor 302 may be coupled to robotic surgery system 300 in alocation where sensor 302 can sense a vibration of surgical tool 312. Asdescribed above, sensor 302 may be integrated directly into armature 310or surgical tool 312. Alternatively, sensor 302 may be mounted directlyto armature 310 of robotic surgery system 300. Sensor 302 may be mountedin an area of robotic surgery system 300 that is outside of a sterilearea.

As described above, sensor 302 is configured to transmit datacorresponding to a sensed vibration. Desirably, sensor 302 solelytransmits data corresponding to vibrations of surgical tool 312. Sensor302 may transmit the data via a wire or wirelessly, as would beunderstood by one of ordinary skill in the art.

As described above, while sensor 302 is described as sensing vibrationsof surgical tool 312, it will be understood that sensor 302 may alsosense audio signals generated by the surgical tool. For example, sensor302 may be a microphone coupled to surgical system 300 such that itsenses audio signals generated from surgical tool 312 contacting thepatient during an operation. Sensor 302 may be configured to transmitdata corresponding to the sensed audio signals.

In step 404, a vibration is provided to the control handle by actuatingan actuator. In an exemplary embodiment, controller 306 actuatesactuator 304 such that actuator 304 provides a vibration to controlhandles 316 when sensor 302 senses a vibration of surgical tool 312.Controller 306 is in communication with sensor 302 to receive data fromsensor 302 corresponding to a sensed vibration of surgical tool 312.Further, controller 306 is in communication with actuator 304 to actuatethe actuators 304 based on the data received from sensor 302. Controller306 may actuate actuators 304 such that actuators 304 provide avibration to control handle 316 when sensor 302 senses a vibration ofsurgical tool 312.

Desirably, controller 306 solely transmits data for actuating actuator304 when sensor 302 senses a vibration of the surgical tool. Controller306 may transmit the data via a wire or wirelessly, as would beunderstood by one of ordinary skill in the art. Controller 306 may onlyactuate actuators 304 when sensor 302 senses vibration occurring withina selected frequency range of vibrations.

As described above, actuator 304 may be integrated directly into controlhandle 316. Alternatively, actuator 304 may be mounted on control handle316 or on a separate structure of control station 314 that is physicallyconnected with control handle 316.

As described above, controller 306 actuates actuator 304 to providevibrations to control handle 316 that correspond to a sensed vibrationfrom surgical tool 312. Desirably, the vibrations provided by actuator304 may be high frequency vibrations, between about 10 Hz and 1000 Hz.Further, the vibrations provided by actuator 304 may match the frequencyof the vibrations of surgical tool 312 sensed by sensor 302.

As described above, while actuators 304 are described as providingvibrations to the control handle, it will be understood that controller306 may actuate actuators 304 such that they provide audio signals tothe operator of control handles 316. For example, actuators 304 may bespeakers, and may be mounted at the control station to provide audiosignals matching the audio signals generated by surgical tool 312 duringan operation.

Method 400 may also include the step of controlling the amplitude ofvibration provided by the actuator to the control handle with anamplitude controller with the controller. In an exemplary embodiment,amplitude controller 308 is electrically connected with controller 306.The operator of control station 314 may control the amplitude of thevibration provided to control handles 316 by the actuator 304 usingamplitude controller 308.

FIG. 9 is a flowchart illustrating an exemplary method 500 forperforming robot-assisted surgery in accordance with an aspect of thepresent invention. As an overview, method 500 includes performing asurgical procedure using a robotic surgery system, generating avibration with a surgical tool coupled to a robotic armature, andreceiving a vibration at a control handle. For the purposes ofillustration, the steps of method 500 are described herein with respectto the components of system 300. Additional details of method 500 aredescribed below.

In step 502, a surgical procedure is performed using a robotic surgerysystem. In an exemplary embodiment, a surgical procedure is performedusing robotic surgery system 300. The surgical procedure may be anyknown surgical procedure including, for example, urologic, gynecologic,cardiac, oral, or plastic surgical procedures. Robotic surgery system300 includes at least one robotic armature 310 for manipulating asurgical tool 312 and a control station 314 having a control handle 316for operating the robotic armature 310. A suitable robotic surgerysystem 300 is the DA VINCI Surgical System, made by Intuitive Surgical,Inc. Other suitable robotic surgery systems will be known to one ofordinary skill in the art from the description herein.

In step 504, a vibration is generated with a surgical tool is coupled toa robotic armature. In an exemplary embodiment, surgical tool 312 iscoupled to robotic armature 310. When an operator of the robotic surgerysystem manipulates the surgical tool 312 during the surgical procedure,surgical tool 312 generates a vibration. Sensor 302 senses a vibrationof surgical tool 312.

As described above, sensor 302 is configured to transmit datacorresponding to the sensed vibration. Desirably, sensor 302 solelytransmits data corresponding to vibrations of surgical tool 312. Sensor302 may transmit the data via a wire or wirelessly, as would beunderstood by one of ordinary skill in the art.

As described above, while surgical tool 312 is described as generatingvibrations, it will be understood that surgical tool 312 may alsogenerate audio signals. Sensor 302 may be configured to sense andtransmit data corresponding to the sensed audio signals.

In step 506, a vibration is received at a control handle of the roboticsurgical system. In an exemplary embodiment, controller 306 actuatesactuator 304 such that actuator 304 provides a vibration to controlhandles 316. The operator of the robotic surgery system thereby receivesa vibration through the operator's contact with the control handles 316during the surgical procedure.

As described above, controller 306 actuates actuator 304 to providevibrations to control handle 316 that correspond to a sensed vibrationfrom surgical tool 312. Desirably, the vibrations provided by actuator304 may be high frequency vibrations, between about 10 Hz and 1000 Hz.Further, the vibrations provided by actuator 304 may match the frequencyof the vibrations of surgical tool 312 sensed by sensor 302.

As described above, while vibrations are received at control handles316, it will be understood that actuators 304 may also generate audiosignals. Actuators 304 may be speakers configured to generate soundscorresponding to the sensed audio signals, which may be received by anoperator of the robotic surgery system 300 at the control station 314.

FIG. 11 is a flowchart illustrating an exemplary method 600 for trainingan operator of a robotic surgery system in accordance with an aspect ofthe present invention. Method 600 may be employed to train an operatorof the robotic surgery system based on vibration feedback detectedduring a test procedure. As an overview, method 600 includes enablingthe operator to perform a procedure, recording vibrations, andgenerating a score for the operator. For the purposes of illustration,the steps of method 600 are described herein with respect to thecomponents of system 300. Additional details of method 600 are describedbelow.

In step 602, the operator is enabled to perform a procedure. Theprocedure may be an actual surgical procedure, or more preferably, theprocedure may be a test procedure used for training the operator. Thetest procedure may be a simulated surgical procedure, or may be anotherprocedure designed to test the operator's dexterity, speed, and/orprecision with the robotic surgery system. Such procedures include, forexample, peg transfer, needle passing, and suturing. In an exemplaryembodiment, the operator is enabled to use robotic surgery system 300 toperform a simulated surgical procedure. Robotic surgery system 300includes at least one robotic armature 310 for manipulating a surgicaltool 312 and a control station 314 having a control handle 316 foroperating the robotic armature 310. Suitable simulated surgicalprocedures will be known to one of ordinary skill in the art from thedescription herein.

During the procedure, a vibration is generated with a surgical toolcoupled to a robotic armature. In an exemplary embodiment, surgical tool312 is coupled to robotic armature 310. When an operator of the roboticsurgery system manipulates the surgical tool 312 during the procedure,surgical tool 312 generates a vibration.

In step 604, vibrations of the surgical tool are recorded. In anexemplary embodiment, sensor 302 senses a vibration of surgical tool 312during the procedure. As described above, sensor 302 is configured totransmit data corresponding to the sensed vibration to controller 306.Desirably, sensor 302 solely transmits data corresponding to vibrationsof surgical tool 312. Sensor 302 may transmit the data via a wire orwirelessly, as would be understood by one of ordinary skill in the art.Upon receiving the data, controller 306 is configured to record thevibrations of the surgical tool. Desirably, controller 306 records thevibrations of surgical tool 312 in conjunction with a time during theprocedure at which those vibrations were sensed by sensor 302.

As described above, sensor 302 may record vibrations of surgical toolalong multiple different axes, as described above. In this case,controller 306 may separately record the vibrations along the multipledifferent axes, or may sum the recorded vibrations (as described above)and record only the summed vibration.

Step 604 may further include recording images and/or video of thesurgical tool during the procedure. In an exemplary embodiment, roboticsurgery system 300 further includes at least one camera 322. Camera 322may be part of a digital video disc (DVD) recorder, and may have thecapacity to record both video and audio during the procedure. Camera 322has a field of view including surgical tool 312. In this embodiment,camera 322 records images and/or video of surgical tool 312 during theprocedure, and transmits data corresponding to the recorded images/videoto controller 306. Camera 322 may transmit the data in any of themanners described above with respect to sensor 302. Upon receiving thisdata, controller 306 is configured to record the images/video of thesurgical tool. Desirably, controller 306 records the images/video ofsurgical tool 312 in conjunction with a time during the procedure atwhich those images/video were recorded by camera 322.

Because haptic vibration signals differ from audio signals only in termsof bandwidth (1,000 Hz vs. 22,000 Hz), stereo audio input channels of aDVD recorder may be used to log the vibrations measured from thesystem's surgical tools. A voltage divider with an attenuation factor of17.5 may be used to adjust each vibration signal to a range of ±1 V tomatch consumer RCA audio input standards. Use of a DVD recorder may bedesirable because they are common in operating rooms and can storehigh-bandwidth vibration data on a compact, durable medium. Furthermore,this choice allows for simultaneous and synchronized capture of thelaparoscopic video stream, which provides context for the instrumentvibration signals during post hoc analysis.

As shown in FIGS. 5A and 6, robotic surgery system 300 includes a pairof armatures 310: a left hand armature and a right hand armatureconfigured to manipulate surgical tool 312. Controller 306 may beconfigured to separately record vibrations of the left hand armature andthe right hand armature. Separately recording these vibrations may bedesirable in order to provide specific training information for theoperator addressing the operator's left hand and/or right handperformance.

In step 606, a score is generated for the operator. In an exemplaryembodiment, controller 306 generates a score for the operator based atleast in part on the recorded vibrations of surgical tool 312.Desirably, controller 306 is configured to automatically generate thescore for the operator based at least in part on the recorded vibrations(i.e., generate the score without human input). Controller 306 maygenerate the operator's score using any one or more of the followingprocesses.

Where step 604 further includes recording images and/or video ofsurgical tool 312, step 606 further includes generating the score basedat least in part on the recorded images/video of surgical tool 312. Forexample, controller 306 may display the recorded video to an experiencedsurgeon, and enable the experienced surgeon to input informationcomprising a store for the operator based on the surgeon's review of thevideo and/or audio (based on his or her own experience). The surgeon mayscore the video recordings of the procedure using conventional ratingscales, such as the Objective Structured Assessment of Technical Skill(OSATS) global rating scale, and/or the Global Evaluative Assessment ofRobotic Skills (GEARS) scale.

Similarly, where step 604 includes separately recording vibrations forthe left hand and right hand armatures, step 606 may further includegenerating the score based on both the recorded vibrations of the leftarmature tool and the recorded vibrations of the right hand armature.For example, the separate vibrations may be subtracted to determinewhich armature is predominantly used during the procedure (i.e., theoperator's handedness). For another example, the separate vibrations maybe multiplied to determine a product of both vibrations (in the sametime step). The times at which the product of the vibrations peaksrepresent times when both armatures were vibrating. These times may beindicative of times during which the armatures are jointly manipulatingthe same object (e.g. surgical tool 312), or are each manipulatingdifferent objects that are in contact with each other.

In one embodiment, controller 306 is configured to generate the scorebased at least in part on the magnitude of the recorded vibrations. Ithas been determined that the magnitude of vibrations may correlate withthe skill of the operator of robotic surgery system 300. In other words,the higher the magnitude vibrations throughout the procedure, the lowerthe skill of the operator; the lower, the higher the skill. Controller306 may desirably calculate the average root-mean-square (RMS) vibrationmagnitude of surgical tool 312 during the procedure to use in generatingthe score for the operator. Additionally, controller 306 may onlyconsider vibrations above a certain minimum amplitude (e.g., 0.2 m/s²)in calculating the total or average RMS vibration magnitude.Accordingly, controller 306 may generate a score for the operatorreflective of the overall magnitude of vibrations recorded during theprocedure.

In another embodiment, controller 306 is configured to generate thescore based on the number of vibration events occurring in the recordedvibrations. It has been determined that the number of times vibrationscross a predetermined minimum threshold during a procedure (i.e. thenumber of vibration events) may correlate with the skill of the operatorof robotic surgery system 300. In other words, the more vibration eventsduring the procedure, the lower the skill of the operator; the less, thehigher the skill. Accordingly, controller 306 may generate a score forthe operator reflective of the overall number of vibration events thatoccurred during the procedure.

In still another embodiment, controller 306 is configured to generatethe score by comparing the recorded vibrations to one or more sets ofstored vibrations, and generating the score based on differences betweenthe recorded vibrations and the one or more stored vibrations. In thisembodiment, controller 306 may pre-record a plurality of sets ofvibrations for operators of varying known skill levels performing thesame procedure on robotic surgery system 300. For example, controller306 may pre-record sets of vibrations from known novice, intermediate,and expert level surgeons. Controller 306 may then compared thevibrations recorded during step 604 to the stored vibrations to see howsimilar/dissimilar the recorded vibrations are to the stored ones. Themore similar the recorded vibrations are, the closer the operator is tothe level of skill of the operator for the pre-recorded vibrations.Accordingly, controller 306 may generate a score for the operatorreflective of the differences between the recorded vibrations and thestored vibrations. Alternatively, where there are a plurality of sets ofstored vibrations, controller 306 may assign the operator a scorematching the level of skill for the closest set of stored vibrations.

Method 600 may also include the step of providing vibrations to theoperator during the procedure that correspond to the vibrations of thesurgical tool. These vibrations may be provided in substantially thesame manner described above with respect to robotic surgery system 300(e.g., by actuating actuators 304 to provide vibrations to controlhandles 316). The operator may be allowed to select the level offeedback provided (e.g. using amplitude controller 308) or may beprovided with a constant, fixed level of feedback during the procedure.In some embodiments, the level of feedback may be selected to beamplified with respect to the actual vibrations occurring during theprocedure, in order to make the operator sensitive to the occurrence ofany vibration during the procedure.

Method 600 may also include the step of providing the score to theoperator following the procedure. In an exemplary embodiment, roboticsurgery system 300 may include an output device in communication withcontroller 306. The output device may be any suitable output deviceknown to one of ordinary skill in the art, including any video or imagedisplay means. Controller 306 is configured to provide the score to theoperator, e.g., by displaying the score to the operator following theprocedure.

Method 600 may further comprise generating trend data for an operator.In an exemplary embodiment, the operator is allowed to repeat the testprocedure one or more times. During each successive procedure,controller 306 records the vibrations of the surgical tool 312.Controller 306 may be configured to generate an average score for thisgroup of procedures, or may generate a score for the operator duringeach repeated procedure. Where controller 306 generates a scorefollowing each procedure, controller 306 may further be configured tocompare the score from each procedure to a previous score, and identifya trend (either improving or degrading) in the scores for the operator.

The above systems and methods are particularly suitable for providingtactile feedback in robotic surgery systems. While tactile feedback mayinclude many different sensations associated with touch, the abovesystems and methods are directed to providing vibrations, and moreparticularly high-frequency vibrations, to the operator of the system.

A surgical tool may experience both forces and vibrations during asurgical procedure. It has been determined that it may be overlydifficult or expensive to realistically implement force feedback to theoperator of a robotic surgery system during the surgical procedure.Additionally, it has been discovered that implementing vibrationfeedback may be equally or more useful than force feedback to theoperator during the surgical procedure. Accordingly, the above systemsand methods implement vibration feedback to convey the events that takeplace during the surgical procedure to the operator.

It will be understood that tactile feedback can serve a criticalfunction during manipulation of a surgical tool during surgery, e.g.,for differentiating tissue types, handling suture needles, and detectingtool-tool collisions. When a robotic surgical system is employed, normalmethods of tactile feedback (i.e. by direct contact between the surgeonand the surgical tool) are lost. Visual feedback employed on roboticsurgery systems may be ineffective or insufficient to make up for thelack of tactile feedback. Thus, the tactile feedback provided by theabove systems and methods may improve the quality of robot-assistedsurgery and help surgeons to more quickly learn how to use roboticsurgery systems.

Additionally, it will be appreciated that systems and methods accordingto this invention can provide substantially real-time vibration feedbackto users of robotic systems so that they can be provided an additionalsensory feedback to guide their use of the robotic system. For example,tactile or auditory or other sensory feedback can be providedcontemporaneously with the actions of the robotic system so that theexperience of the user is concurrent with interactions of the robot withits environment. Such additional sensory feedback adds to the richnessof the user's experience and allows optimized use of the robotic system.

Such sensory feedback is optionally selectable by a user. Depending on aparticular user's preference, the additional sensory feedback can beactivated, deactivated, or changed in magnitude to suit individualpreferences.

Also, systems and methods according to this invention can providevibration feedback from a wide variety of interactions between therobotic system and a subject being manipulated by the system. Forexample and purposes of illustration, in the context of a surgicalsystem, the system can provide vibration feedback from the operation ofone or more surgical instrument (e.g., opening or closing a grip),interaction among instruments (e.g., tool-to-tool contact), and contactbetween instruments and other objects in the surgical field (e.g.,contacting or releasing or grabbing a suture or tissue or needle).

Although the invention is illustrated and described herein withreference to specific embodiments, the invention is not intended to belimited to the details shown. Rather, various modifications may be madein the details within the scope and range of equivalents of the claimsand without departing from the invention.

What is claimed:
 1. A method for training an operator of a roboticsurgery system, the method comprising: enabling the operator to performa test procedure; recording vibrations of a surgical tool of the roboticsurgery system during the test procedure; and generating a score for theoperator based at least in part on the recorded vibrations of thesurgical tool.
 2. The method of claim 1, wherein the test procedure is asimulated surgical procedure.
 3. The method of claim 1, wherein therecording step further comprises recording video of the surgical tool ofthe robotic surgery system during the test procedure, and the generatingstep comprises generating a score for the operator based on the recordedvideo of the surgical tool.
 4. The method of claim 1, wherein therecording step comprises: recording vibrations along multiple differentaxes.
 5. The method of claim 4, wherein further comprising the step of:summing the vibrations along the multiple different axes to generate asummed vibration.
 6. The method of claim 1, wherein the robotic surgerysystem comprises a left hand armature and a right hand armature, andwherein the recording step comprises: separately recording vibrations ofthe left hand armature and the right hand armature.
 7. The method ofclaim 6, wherein the generating step comprises: generating the scorebased on a product of the recorded vibrations of the left hand armatureand the recorded vibrations of the right hand armature.
 8. The method ofclaim 1, wherein the step of generating the score comprisesautomatically generating the score.
 9. The method of claim 1, whereinthe generating step comprises: generating the score based at least inpart on a magnitude of the recorded vibrations.
 10. The method of claim1, wherein the generating step comprises: generating the score based onthe number of vibration events occurring in the recorded vibrations. 11.The method of claim 1, wherein the generating step comprises: comparingthe recorded vibrations to one or more stored vibrations; and generatingthe score based on differences between the recorded vibrations and theone or more stored vibrations.
 12. The method of claim 1, furthercomprising the step of: providing vibrations corresponding to thevibrations of the surgical tool to the operator during the testprocedure.
 13. The method of claim 1, further comprising the step of:providing the score to the operator following the test procedure. 14.The method of claim 13, further comprising the step of: enabling theoperator to repeat the test procedure; and recording vibrations of thesurgical tool during the repeat test procedure.
 15. The method of claim14, further comprising: generating a score for the operator based atleast in part on the recorded vibrations of the surgical tool during therepeat test procedure; comparing the score for the repeat test procedureto the score for the test procedure; identifying a trend in scores forthe operator.
 16. A robotic surgery system comprising: an armatureconfigured for manipulating a surgical tool; a control station,positioned remote from the armature, having a control handle configuredfor operating the armature; a sensor positioned to sense a vibration ofthe surgical tool; and a controller in communication with the sensor,the controller being configured to record vibrations of the surgicaltool during a test procedure performed by an operator, and generate ascore for the operator based at least in part on the recorded vibrationsof the surgical tool.
 17. The system of claim 16, further comprising atleast one camera having a field of view including the surgical tool,wherein the controller is further configured to record video of thesurgical tool during the test procedure, and generate a score for theoperator based on the recorded video of the surgical tool.
 18. Thesystem of claim 16, wherein the controller is configured to recordvibrations along multiple different axes.
 19. The system of claim 18,wherein the controller is further configured to sum the vibrations alongthe multiple different axes to generate a summed vibration.
 20. Thesystem of claim 16, wherein the robotic surgery system comprises a lefthand armature and a right hand armature configured to manipulate thesurgical tool, and wherein the controller is configured to recordvibrations of the left hand armature and separately record vibrations ofthe right hand armature.
 21. The system of claim 20, wherein thecontroller is configured to generate the score based on a product of therecorded vibrations of the left hand armature and the recordedvibrations of the right hand armature.
 22. The system of claim 16,wherein the controller is configured to automatically generate thescore.
 23. The system of claim 16, wherein the controller is configuredto generate the score based at least in part on a magnitude of therecorded vibrations.
 24. The system of claim 16, wherein the controlleris configured to generate the score based on the number of vibrationevents occurring in the recorded vibrations.
 25. The system of claim 16,wherein the controller is configured to compare the recorded vibrationsto one or more stored vibrations, and generate the score based ondifferences between the recorded vibrations and the one or more storedvibrations.
 26. The system of claim 16, further comprising an actuatorpositioned to provide a vibration to the control handle of the controlstation, wherein the controller is further configured to providevibrations corresponding to the vibrations of the surgical tool to theoperator during the test procedure using the actuator.
 27. The system ofclaim 16, further comprising an output device in communication with thecontroller, wherein the controller is further configured to provide thescore to the operator following the test procedure.